Content Filtering support is an enhanced, or extended, premium service. The System Administration Guide provides basic system configuration information, and the product administration guides provide procedures to configure basic functionality of core network service. It is recommended that you select the configuration example that best meets your service model, and configure the required elements for that model, as described in the respective product Administration Guide, before using the procedures in this chapter.
|
l
|
Content Filtering is an in-line service available for 3GPP and 3GPP2 networks to filter HTTP and WAP requests from mobile subscribers based on the URLs in the requests. This enables operators to filter and control the content that an individual subscriber can access, so that subscribers are inadvertently not exposed to universally unacceptable content and/or content inappropriate as per the subscribers’ preferences.
In the URL Blacklisting solution, all HTTP/WAP URLs in subscriber requests are matched against a database of “blacklisted” URLs. If there is a match, the flow is discarded, redirected, or terminated as configured. If there is no match, subscribers view the content as they would normally.
URL Blacklisting may/may not be a subscriber opt-in service, operators can enable URL Blacklisting either for all subscribers or for a subset of subscribers. Typical cases include applying a blacklisted database of child porn URLs to all subscribers so that they are inadvertently not exposed to such universally unacceptable content.
In Category-based Static Content Filtering, all HTTP/WAP URLs in subscriber requests are matched against a static URL categorization database. Action is taken based on a URL’s category, and the action configured for that category in the subscriber’s content filtering policy. Possible actions include permitting, blocking, redirecting, and inserting/altering content.
In Category-based Static-and-Dynamic Content Filtering, if static rating categorizes a URL as either “dynamic” or “unknown”, the “requested content” is sent for dynamic rating. Wherein the “requested content” is analyzed and categorized. Action is taken based on the category determined by dynamic rating, and the action configured for that category in the subscriber’s content filtering policy. Possible actions include permitting, blocking, redirecting, and inserting/altering content.
Typically Category-based Content Filtering is an opt-in service, subscribers self-choose a content-filtering policy or plan, such as Teen, Child, Adult, etc., and are subjected to content filtering as per their chosen plan. Also, the content-filtering policies of different subscribers may be different, enabling differential access of content to them. This solution provides maximum flexibility, and is also referred to as the Policy-based Content Filtering.
Both URL Blacklisting and Category-based Content Filtering support can be concurrently enabled on a system.
Content Filtering uses Deep Packet Inspection (DPI) feature of Enhanced Charging Service (ECS) to discern HTTP and WAP requests.
Content Filtering is an in-line service available for all ST-series Multimedia Core Platforms running core network services.
[600-00-7801] Blacklisting Integrated Service
[600-00-7586] Integrated Content Filtering Service, 1k Sessions
IMPORTANT: External Content Filtering Server support through Internet Content Adaptation Protocol (ICAP) interface is a licensed feature, requiring a separate license. For more information, see the ICAP Interface Support chapter of the System Enhanced Feature Configuration Guide.IMPORTANT: For information on obtaining and installing licenses, refer to Managing License Keys in the System Administration and Configuration Guide.In the URL Blacklisting solution, a blacklist is a list of known URLs/URIs, which for some reason are being denied recognition. The blacklist can be obtained from a known source such as the National Center for Missing & Exploited Children (NCMEC, http://www. missing kids.com), or any other IP source. The blacklist is a clear text file, the file must be named cumulative.csv, and must use the same format as the blacklist file from NCMEC. For more information on the blacklist file please contact your local service representative.
Unlike the Category-based Content Filtering solution, which categorizes URLs as per a static database and takes different actions based on the different policies associated with subscribers, URL Blacklisting is applicable to all subscribers associated with a blacklisting-enabled rulebase. The same blacklist database is used for all subscribers, and for a specific URL, the same action is taken for all subscribers.
The blacklist file is downloaded and converted into a non human-readable optimized format (OPTBLDB) and then made available in the system. Once in place, all HTTP and WAP requests from subscribers are inspected in order to determine the requested destination URL/URI. If the URL/URI is not present in the blacklist then the request is passed on as usual. If the URL/URI is present in the blacklist, the request is dropped, or the flow is redirected or terminated as configured. There is no indication/messaging sent to the requesting subscribers that the requested HTTP/WAP URL/URI was rejected due to a blacklist match.
The URL Blacklisting match-method can be configured to either be generic or to look for any URL/URI in its exact, literal form.
The system generates usage/event data that can be utilized as the basis for blacklist reporting. The offline reports consist of, at a minimum, a running total of the number of times a match was made against the blacklist without any information regarding the specifics of the request.
The default/configured number of versions of the Blacklist database are maintained on the chassis (both the SPCs). This enables reverting to a particular version if required.
The following figure shows the high-level URL Blacklisting architecture with ECS, and other components in a deployment scenario.
The URL Blacklisting solution uses the deep-packet inspection capabilities of ECS for URL/URI extraction.
|
l
|
Session Controller (SessCtrl): The SessCtrl runs on the primary SPC/SMC and is responsible for managing ECS and URL Blacklisting services.
|
|
l
|
Session Manager (SessMgr): A single SessMgr treats ECS charging and URL Blacklisting that is applicable to common subscriber sessions.
|
The WEM is a server-based application enabling complete element management of the system. The UNIX-based server application works with the network elements within the system using the Common Object Request Broker Architecture (CORBA) standard.
IMPORTANT: For information on WEM administration, refer to the Web Element Manager Installation and Administration Guide.|
l
|
Downloads the URL Blacklist database (cumulative.csv) from the specified source at configured schedule
|
|
l
|
Converts the URL Blacklist database (cumulative.csv) file to Starent Format Master Database (SFMDB) file
|
The URL Blacklisting solution includes the CF-CDP consisting of a geographically-redundant server hosting the CDP application that may be the same as the External Storage Server in ESS.
In the URL Blacklisting solution, the CF-CDP serves as a repository and distribution node for the Blacklisting database and its updates (OPTCMBLDB). There are no incremental updates in the URL Blacklisting feature as a new Full database will be supplied by WEM daily.
|
1
|
The WEM downloads the blacklist file from the specified source (NCMEC/other). The clear text file is converted into a non-human readable optimized format (OPTBLDB) and then pushed to the CDP.
|
|
2
|
The CDP pushes the optblk.bin file to the chassis (to the flash/pcmcia device) at pre-determined intervals. The optblk.bin file contains the full blacklist. If this file is verified to be correct it replaces the optblk.bin file on the chassis, and the last optblk.bin is rolled over.
|
|
3
|
The blacklist file is auto-detected by the Session Controller (SessCtrl), which verifies the integrity of the Blacklist database using checksums, and then loads it.
|
The new blacklist is loaded only if it has been received properly. If the full Blacklist database is not found, corrupted, or if the loading fails, traps are generated. Correspondingly clear traps are also generated on a valid Blacklist database being available, and after a successful load.
IMPORTANT: The URL Blacklisting feature is enabled only if the url-blacklisting action is set in any of the rulebases. Thus, the automatic detection of the Blacklist database, storing it in memory, and loading onto the SessMgrs will happen only if the url-blacklisting action is set in any of the rulebases. |
5
|
The Blacklist database is loaded on each SessMgr as and when they come up (if URL Blacklisting is set in any rulebase) or when URL Blacklisting gets set in any of the rulebases.
|
When the SessMgrs start for the first time or after recovery, if URL Blacklisting is set in any of the rulebases, the stored Blacklist database at SessCtrl is loaded onto the SessMgrs. This holds true for standby managers as well i.e., when standby managers come up the Blacklist database is loaded onto them.
Whenever a SessMgr is killed, standby manager which already has the Blacklist database loaded takes its place, and a new standby manager is created which loads the Blacklist database as part of SessMgr getting started for the first time.
If SessCtrl is killed, while recovering it checks if URL Blacklisting is set in any of the rulebases, if set it will store the Blacklist database onto itself and load all the SessMgrs as well.
|
6
|
When a new Blacklist database is loaded on to the SessMgrs, the new database (and any stored versions that have rolled over) are synced to the other SPC so that after switchover, the proper Blacklist database can be accessed.
|
|
1
|
When an initial HTTP/WAP request comes for ECS processing and is processed by the ACS subsystem, a check is made to see if the URL Blacklisting support is enabled.
|
|
2
|
If enabled, the URL is extracted from the incoming request and is matched with the local in-memory Blacklist database.
|
If a match is found for the URL in the Blacklist database, the packets are treated as per the blacklisting action configured—Discard, Redirect, or Terminate flow.
In case of multiple HTTP requests in the same TCP packet, if any of the URLs match the packet is treated as per the blacklisting action configured.
The Category-based Content Filtering application is a fully integrated, subscriber-aware in-line service provisioned on ST-series Multimedia Core Platform running HA services. This application is transparently integrated within the ECS, and utilizes a distributed software architecture that scales with the number of active HA sessions on the system.
Content Filtering policy enforcement is the process of deciding if a subscriber should be able to receive some content. Typical options are to allow, block, or replace/redirect the content based on the rating of the content and the policy defined for that content. For the list of content categories, refer to the Category List appendix in the Content Filtering Services Administration Guide.
The Category-based Content Filtering solution enables operators to ensure a simplified end-to-end traffic flow with a simple network topology. In-line deployment of Content Filtering provides a more attractive solution in contrast to out-of-line solutions where the filtering and policy enforcement is provided at some offload point that is decoupled from the bearer-processing layer.
The out-of-line model forces a session to make multiple hops through a redundant array of equipment which has a negative impact on traffic latency and limits subscriber and network visibility. In addition, the out-of-line model requires all subscriber sessions to be steered to the adjunct Content Filtering platform for policy enforcement regardless of whether this additional processing is needed. This leads to increased bandwidth provisioning requirements on gateway routers.
To facilitate network simplicity, it makes sense to leverage the benefits of deep packet inspection at a single policy enforcement point that is tied to the bearer processing layer. The advantages of this approach implemented in include the following benefits:
|
l
|
Reduced processing latency: In-line service processing eliminates unnecessary hand-offs and forwarding to external network elements.
|
|
l
|
Simplified policy provisioning: Enables all policies like Content Filtering, ECS and QoS to be retrieved from same AAA/Policy Manager signaling interface thus reducing total volume of control transactions and associated delay.
|
|
l
|
Simplified provisioning and complete service integration: Provisioning of separate resources like PACs/PSCs for processing of subscriber data sessions and discrete services are eliminated. The same PAC/PSC CPU can contain active Session Manager tasks for running Content Filtering and ECS charging.
|
|
l
|
Integration with Content Service Steering (CSS) architecture: Enables applicable sessions to be forwarded to the in-line content filtering subsystem while delay and time sensitive voice/multimedia services immediately forwarded to Internet.
|
|
l
|
Service control: Precise control over the interaction and service order handling of bearer flows with required applications like Content Filtering, ECS, Subscriber-aware Stateful Firewall, integrated Policy Charging and Rules Function (PCRF) for Service Based Bearer Control.
|
Apart from the advantages described previously, Category-based Content Filtering service reduces the requirement of over-provisioning of capacity at neighboring gateway routers. It also eliminates requirements of external Server Load Balancers and enhances the accuracy in subscriber charging records.
With Static Category-based Content Filtering, the filtering is only as good as the collection of URLs in the database. Even the largest URL database covers only a fraction of the Surface Web and virtually none of the Deep Web. It is quite impossible to find, review, and categorize enough of the available Web sites to keep the database current.
Also, many mobile sites are classified as dynamic sites. A dynamic site may return either acceptable or inappropriate content from the same URL. For example, search engines, news portals, or auction sites that return variable results depending upon subscriber requests.
When the Content Filtering subsystem receives a request for dynamic content it becomes necessary to categorize pages in real-time to determine how to classify content the provider is delivering at that moment. The “Static Rating” solution that relies exclusively on previously categorized rating for sites may fail to categorize dynamic sites appropriately.
Dynamic Content Filtering enables on-the-fly content analysis of Web traffic using different content analysis techniques. When a Web page is received, it is analyzed and then categorized according to the content found in the page. Whether a Web site has existed for five months or for five minutes does not matter since determination of the category to which the Web page belongs is made just at the time of request. Therefore, dynamic filters have no problem keeping up with the growth and changing content of the Internet. A combination of static filtering and dynamic inspection provides real accuracy and scalability as the Web weaves an increasingly sophisticated network of sites.
IMPORTANT: Category-based Content Filtering can only work in static-only or in static-and-dynamic modes. Dynamic-only Content Filtering mode is not supported.In Static-and-Dynamic Content Filtering every URL will first undergo static rating, if the URL cannot be rated by the static database, or if the URL’s statistic rating is categorized as DYNAM, then it will go for dynamic rating. After the content has been analyzed, as with static content filtering, dynamic rating actions include acceptance, blocking, redirection, and/or replacement of content.
Static-and-Dynamic Content Filtering must be enabled at the global and rulebase levels. Before enabling static-and-dynamic rating in the rulebase, it must be enabled at the global level as the resources required for dynamic rating are allocated at the global level. When enabled in a rulebase, it is applied for subscribers using that rulebase.
|
l
|
Dynamic rating is accurate only for complete response data and does not work properly for data divided in packets. Since only first response packet is used for dynamic rating and not the complete response, the rating will be only 60-80% accurate.
|
|
l
|
Only one category “PORN” is supported in all languages, while 14 other categories are supported in English.
|
The Category-based Content Filtering solution is provided as an integrated subsystem within the Enhanced Charging Service (ECS). Although it is not necessary to provision content-based charging in conjunction with content filtering, it is highly desirable as it enables a single point of deep-packet inspection for both services. It also enables a single policy decision and enforcement point for both services thereby streamlining the required number of signaling interactions with external AAA/Policy Manager servers. Utilizing both services also increases billing accuracy of charging records by insuring that mobile subscribers are only charged for visited sitescontent.
The Category-based Content Filtering solution uses Content Filtering Policy to analyze the content requested by subscribers. Content Filtering Policy provides a decision point for analyzed content on the basis of its category and priority.
The Category-based Content Filtering solution also utilizes ACS rulebases in order to determine the correct policy decision and enforcement action such as accept, block, redirect, or replace. Rulebase names are retrieved during initial authentication from the AAA/Policy Manager. Some possible examples of rulebase names include Consumer, Enterprise, Child, Teen, Adult, and Sport. Rulebase names are used by the ACS subsystem to instantiate the particular rule definition that applies for a particular session. Rulebase work in conjunction with a content filtering policy and only one content filtering policy can be associated with a rulebase.
IMPORTANT: For more information on rulebases and rule definitions, refer to the Enhanced Charging Services Administration Guide.The ACS subsystem includes L3–L7 deep packet inspection capabilities. It correlates all L3 packets with higher layer criteria such as URL detection within an HTTP header, it also provides stateful packet inspection for complex protocols like FTP, RTSP, and SIP that dynamically open ports for the data path.
The Content Filtering subsystem uses the deep-packet inspection capabilities of ECS for URL/URI extraction.
|
l
|
Session Controller (SessCtrl): The SessCtrl runs on the primary SPC/SMC and is responsible for managing ECS and Content Filtering services.
|
|
l
|
Session Manager (SessMgr): A single SessMgr treats ECS charging and Content Filtering that is applicable to common subscriber sessions.
|
The following figure shows a high-level view of the Category-based Content Filtering architecture with ECS, and other components in a deployment scenario.
Figure 1-2 shows these components and their location in the high-level Category-based Content Filtering architecture.
This is an internal categorization database (periodically synchronized with an external server) that provides ratings for publicly accessible traditional and mobile Web sites. When the SessMgr passes a URL/URI to internal list server, the list server returns a list of matching category ratings.
The list server is used to determine whether a Web site has already been classified. When the list server passes back a category rating to the filtering application, the rating is compared against the Category Policy ID applied for the subscriber to determine the appropriate action like accept, block, redirect, or replace. If the list server returns a clean rating, there is no need to perform a real-time analysis of any content delivered by the site.
When a blocked or rejected content rating is returned, the SessMgr can insert data such as a redirect server address into the bearer data stream. If no rating is returned this means the site is capable of returning either clean or unacceptable content. In this case, the Content Filtering application uses the real-time dynamic analysis engine to examine additional content served by the site.
Each SRDB contains a replication object consisting of hash tables that map known Web sites and their subdirectories to their respective category ratings. The SessCtrl reads the index of SRDB tables with a data structure that associates keys with URL rating values and loads it onto the SRDB managers.
To boost performance and provide high availability, SRDB Manager provides functionality to load the Optimized Content Rating Master Database (OPTCMDB) volumes from its peer SRDB task. If the peer SRDB task is not in loading state then the OPTCMDB loading is done through SessCtrl to the recovered SRDB task.
When Static-and-Dynamic Content Filtering is enabled, Dynamic SRDB tasks are spawned based on available PACs/PSCs.
IMPORTANT: To support dynamic rating, a minimum of three active PACs/PSCs are required, that will have one Dynamic SRDB. The number of Dynamic SRDBs may increase with an increase in the number of PACs. The load for rating dynamic responses is distributed equally across all the Dynamic SRDBs created.First one Dynamic SRDB task is created with two Standby SRDBs on the same CPU, then eight Static SRDBs are created, and then more Dynamic SRDB tasks are created if memory is available. A Dynamic SRDB is always created with two Standby SRDBs with it on the same CPU.
The Dynamic Rater Package, which contains the model files (used for language detection and category recognition) and feature counters (used to decide whether or not to evaluate the Web page against the respective model file) are loaded on the SRDB Managers. The rater package is loaded only on the active SRDB and not on the standbys.
The Rater package containing the model files (used for language detection and category recognition) and feature counters (used to decide whether or not to evaluate the Web page against the respective model file) is stored at pcmcia1/cf. After loading the static database, ACS will read the Rater package and load it onto the Dynamic SRDB Managers.
The Rater package will also be loaded on SRDBs on recovery/reconciliation if static-and-dynamic Content Filtering is enabled.
The rater package loaded onto SRDBs can be upgraded using an upgrade CLI, that will look for the upgrade file in the form “rater_f.pkg” at a specific location and if found load the new package onto the SRDBs. On successful loading, the “rater_f.pkg” is replaced with “rater.pkg” and versioned. In case of loading or upgrade failures, appropriate traps are generated.
The real-time analyzer requires a model file that defines the features which are necessary to classify a Web page as belonging to a specific category and language. A model file per category is created by analyzing the traits of thousands of pages of that category and thousands of pages that does not belong to that category. For some categories, a feature counter file is used to decide whether or not to evaluate the Web page against the respective model file.
When URL Blacklisting solution is the only content filtering enabled on a system, no SRDB tasks are spawned at startup. Only when either Category-based Content Filtering is enabled in isolation, or with URL Blacklisting, the SRDB tasks are spawned.
This is a third-party content rating solution for exporting content filtering rules database information to the Category-based Content Filtering system. In addition, while exporting database updates, it collects reports of URLs processed by ECS and Content Filtering services that are reported as unknown in the deployed static rating database. This server analyzes these URLs and provides the rating in future updates for static rating database.
This server provides the following supports to Master Content Rating Database Server (MCRDBS) for the content rating function:
|
l
|
Provides full Vendor Format Master Database files (VFMDB) to Master Content Rating Database (MCRDB) server on request from MCRDBS.
|
|
l
|
Provides incremental Vendor Format Master Static URL Database file (VFMDB-INC) to MCRDBS when any incremented VFMDB is available and requested from MCRDBS.
|
|
l
|
Imports the Unknown URLs file (Vendor Format Unknown Database File (VFUNKDB)) from MCRDBS.
|
|
l
|
Exports Vendor Format Dynamic Rating database to Master Content Rating Database Server (MCRDBS) for use with the Third-party Dynamic Content Rating Support.
|
The Category-based Content Filtering solution provides a Master Content Rating Database Server to convert the VFMDB to Starent Networks’ Format Master Database (SFMDB). It handles both full and incremental updates and processes them on a configured schedule.
This sever is also responsible for distribution of Starent Networks’ format data files to WEM servers in the Starent Networks’ customer support infrastructure on a configured interval.
|
l
|
Database conversion: Converts VFMDB files to SFMDB files. It also handles the incremented and unknown database files.
|
|
l
|
Database poller: Provides the converted SFMDB database files to WEM in a preconfigured path.
|
The local and remote external storage servers are part of ECS Storage System in the ECS solution architecture.
The L-ESS and R-ESS are ECS applications running on redundant highly available servers that collect and process EDRs and UDRs from which billing events and reports are generated. Either the system pushes the EDR/UDR files to the L-ESS, or the L-ESS fetches them from the system and processes them into formats suitable for billing mediation servers, Report Engine at CDP, and the R-ESS. The R-ESS consolidates the processed EDR/UDR files into a database for report generation through UDR Report Generator or EDR Report Generator at CDP.
When Content Filtering is deployed in conjunction with ECS, the operator has an option of collocating the Central Decision Point (CF-CDP) with an ESS thereby eliminating the requirement of the CF-CDP to fetch a copy of the CF-EDRs from each system running ECS and Content Filtering service. The database generated on an ESS by processing EDR/UDR records is a superset of the database required by a CF-CDP.
IMPORTANT: For more information on External Storage Systems, refer to the ESS Installation and Administration Guide.The function of the RADIUS Server/Policy Manager in the Content Filtering solution is to provide per-subscriber Content Filtering provisioning information when a subscriber’s session is established. It can also issue a Change-of-Authorization (CoA) to update an in-progress session to modify the Content Filtering policy for a subscriber.
The following are the basic functions provided by a RADIUS Server/Policy Manager in the Content Filtering solution:
|
l
|
|
l
|
Support for Content Filtering Policy identifier VSA to select the content filtering policy within the selected rulebase for a subscriber
|
|
l
|
Support exporting a subscriber provisioning record based on MSID to the customer service interface (Customer Care Interface) so that operator’s customer care executive can see the provisioned content filtering policy for a subscriber
|
The Content Filtering solution provides a GUI to provide information to support staff in the operator’s customer-care support center. This interface allows support personnel to quickly address subscribers questions or concerns about policies on their account.
|
l
|
The ability for the operator to enter a URL and have it content rated as static-and-dynamic
|
|
l
|
Display the list of URLs recently accessed by the subscriber with filtering/sorting the display by time ranges, rating category, URL sub-string, and many more. The data that is displayed is generated by querying the database in all of the CF-CDPs using the subscriber’s IMSI/MSID/Subscriber number as the key.
|
|
l
|
The WEM is a server-based application providing complete element management of the system. The UNIX-based server application works with the network elements within the system using the Common Object Request Broker Architecture (CORBA) standard.
IMPORTANT: For information on WEM administration, refer to the Web Element Manager Installation and Administration Guide.|
l
|
Internet: To communicate with the Master Content Rating Database Server (MCRDBS) which provides update files.
|
|
l
|
Configures and manages the operator-defined White/Black static rating database (WBLIST) for the network (WBLIST is maintained in SFMDB format)
|
|
l
|
Imports full and incremental Starent Format Master Database (SFMDB and SFMDB-INC) files from the MCRDBS on a configured schedule
|
|
l
|
Processes incremental Starent Format Master Database (SFMDB-INC) updates from MCRDBS maintaining an updated SFMDB file
|
|
l
|
Merge the operator’s WBLIST database with the most recent SFMDB creating a Starent Format Customer Master Database (SFCMDB)
|
|
l
|
Computes an incremental update to the OPTCMDB-INC suitable for updating the Content Filtering subsystem that contains a previous version OPTCMDB
|
The Category-based Content Filtering solution also includes a Central Decision Point (CF-CDP) with Report Engine that can be integrated in an R-ESS of ECS Storage System. The CF-CDP consists of a geographically redundant server hosting the CDP application that may be the same as Remote-External Storage Server in ESS.
In the Category-based Content Filtering solution, the CF-CDP performs the following major functions:
The primary activity of the CF-CDP is to process CF-EDRs into a database that supports report generation by the WEM, and query processing by the Customer-Care Management application.
In addition, the CF-CDP serves as a repository and distribution node for the optimized content rating master database and incremental updates (OPTCMDB/OPTCMDB-INC). CF-System uses the OPTCMDB to perform the actual Content Rating Function on subscriber traffic. The OPTCMDB database is also used by an instance of the static and dynamic rating engines running on the CF-CDP to provide a functional rating service that is leveraged by the Customer-Care application to display the static rating of a URL and/or corresponding dynamic rating for the URL’s content.
The CF-CDP server provides updated configuration files to the CF-System with the latest revisions to the static categorization database. The Content Filtering application also provides a mechanism to properly distinguish between release versions. The configuration updates are securely transmitted via SFTP over SSH via the out-of-band management network to a SPIO interface and the local management context of the chassis.
Updates to the CF-System occurs on requests or configured periodicity. To further reduce the volume of traffic over the management network, instead of retransmitting the entire SRDB at each update, it is also possible to send small incremented differential files that include only the additional URLs that were added since the previous update.
The Report Generator utility in CF-CDP is a script-based tool responsible for report generation and CF-EDR parsing. A script-based utility generates reports in XML format for content filtering subscribers and the Report Engine server takes care of EDR parsing. The script can be used with cron job for periodic report generation in background. Reports can also be generated from the WEM GUI.
CF-EDR files are pushed from L-ESS to CF-CDP server at a configured time interval and stored in specified data directory on the CF-CDP server.
The Report Engine generates the following types of reports in XML format on the basis of user parameters, event records, and criteria.
|
l
|
Overall Summary Report: This is a short summary report of all the activities done between a duration of time. This report includes following schema for subscriber activity summary report:
|
|
l
|
Subscriber Detailed Report: This is a detailed report in which operators get detailed information about subscriber activities on a Content Filtering system. This report includes all information about a subscriber’s request with following schema for each URL requested:
|
|
l
|
URL Summary Report: This is a high-level report which provides the list of URLs and the number of times a URL was visited with the following additional schema along with the schema of Summary Report:
|
The Content Filtering Subsystem integrated into the ACS subsystem consists of two primary elements; an onboard static categorization database and a dynamic rating engine. The filtering service uses the Deep Packet Inspection (DPI) capabilities of the ACS subsystem to classify and partition application or protocol specific flows into virtual sessions.
Content analyzers are used to identify various types of flows such as HTTP, MMS/WAP, and POP3 E-mail. A typical HTTP request for a Web page, for example, invokes TCP and HTTP traffic analyzers. Any HTTP field including URLs or URIs can be identified. When a subscriber session is bound by CSS to an ECS running content filtering service, the URL/URI is extracted and compared against the static categorization database.
The following figure and the steps describe how Category-based Content Filtering works during a subscriber call:
|
4
|
AAA server processes the AAA Access Request from the Content Filtering subsystem to create the session, and the Policy Manager in AAA server uses subscriber identification parameters including NAI (username@domain), Calling Station ID (IMSI, MSID) and Framed IP Address (HoA) as the basis for subscriber lookup.
|
|
5
|
The Policy Manager and AAA generate and send an Access Accept message including all policy and other attributes to establish the session to the Content Filtering subsystem.
|
|
l
|
Filter ID or Access Control List Name: Applied to subscriber session. It typically contains the name of the Content Service Steering (CSS) ACL. The CSS ACL establishes the particular service treatments such as Content Filtering, ECS, Traffic Performance Optimization, Stateful Firewall, VPN, etc. to apply to a subscriber session and the service order sequence to use in the inbound or outbound directions. Real-time or delay sensitive flows are directly transmitted to the Internet with no further processing required. In this case, no CSS ACL or Filter ID is included in the Access Response.
|
|
l
|
SN-CF-Category-Policy: Applied to the subscriber content flow. Policy ID included in this attribute overrides the policy identifier applied to subscriber through rulebase or APN/Subscriber configuration. This content filtering policy determines the action to be taken on a content request from subscriber on the basis of its category. At anytime only one content filtering policy can be associated with a rulebase.
|
|
l
|
SN1-Rulebase Name: This custom attribute contain information such as consumer, business name, child/adult/teen, etc.). The rulebase name identifies the particular rule definitions to apply. Rulebase definitions are used in ECS as the basis for deriving charging actions such as prepaid/postpaid volume/duration/destination billing and charging data files (EDRs/UDRs). Rulebase definitions are also used in content filtering to determine whether a type of user class such as teenagers should be permitted to receive requested content belonging to a particular type of category such as adult entertainment, gambling or hate sites. Rulebase definitions are generated in the Active Charging Configuration Mode and can be applied to individual subscribers, to domains or on per-context basis.
|
|
6
|
Content Filtering subsystem creates a new session for MS, and sends the rulebase to ACS subsystem if required.
|
|
11
|
Within the system access control list (ACL) processes the request and directs the request to ECS/Content Filtering subsystem based on the subscriber configuration.
|
Within the system, if the bearer flow is treated by Content Filtering or other in-line services, the SessMgr feeds it to the Content Service Steering (CSS) API. If Content Filtering is the first service touch point, TCP and HTTP traffic analyzers within a given SessMgr utilize deep-packet inspection to extract the requested URL.
When only Static Content Filtering is enabled, first the URL is looked-up in the cache maintained at SessMgr for static URL requests, if there is a hit, the category is returned, if its a miss, a URL look-up is performed by an onboard SRDB for static rating.
|
l
|
If a category is returned, action is taken as configured for that category in the subscriber’s Content Filtering policy:
|
|
l
|
allow: If the category is permitted by the subscriber’s content filtering policy, the request is sent to the server, and the response transmitted to the subscriber’s mobile.
|
|
l
|
content-insert: The system notifies the subscriber’s mobile of the blocked content by inserting a specified message within the IP data stream, and prevents access to the requested content. The insert string is as specified in the subscriber’s content filtering policy.
|
|
l
|
redirect-url: The system inserts a specified redirect server address in the bearer data stream and returns an HTTP error message to the subscriber’s mobile. The redirect address is as specified in the subscriber’s content filtering policy.
|
The redirect server may prompt the subscriber to send additional security credentials in order to access the requested content.
|
l
|
terminate-flow: The system gracefully terminates the TCP connection between the subscriber and server, and sends a TCP FIN to the subscriber and a TCP RST to the server.
|
|
l
|
www-reply-code-and-terminate-flow: The system terminates the flow with a specified reply code to the subscriber’s mobile. The reply code is as specified in the subscriber’s content filtering policy.
|
|
l
|
If a category is not returned / the URL is not present in the database, the system takes the action as configured for the UNKNOW category in the subscriber’s Content Filtering policy.
|
|
l
|
If for the category returned there is no action configured in the subscriber’s content filtering policy, the default action is taken.
|
|
l
|
All URL requests first go for static rating. If the category returned is not DYNAM or UNKNOW, action is taken as configured for that category in the subscriber’s Content Filtering policy.
|
IMPORTANT: URLs present in the static database but without a category associated, which are internally categorized as NONE, are not considered for dynamic rating. These URLs could not be categorized and are considered as unobjectionable sites.|
l
|
If the category returned is DYNAM or UNKNOW (URL not present in the static database), the request is sent to the server. When the response arrives it is sent to any one of the Dynamic SRDBs for dynamic rating.
|
When a request must be sent for dynamic rating, only a single packet in which the HTTP header is complete is sent to any one of the dynamic SRDBs for dynamic rating.
Before sending the response for dynamic rating, the “content-type” is checked and sent only if the content-type is one of the following:
|
l
|
If a category is returned, action is taken as configured for that category in the subscriber’s Content Filtering policy:
|
|
l
|
allow: If the category is permitted by the subscriber’s policy, the requested content is transmitted to the subscriber’s mobile.
|
|
l
|
content-insert: The system replaces the content by inserting a specific message, and prevents access to the requested content. The insert string is as specified in the subscriber’s content filtering policy.
|
|
l
|
redirect-url: The system inserts a specified redirect server address in the bearer data stream and returns an HTTP error message to the subscriber’s mobile. The redirect address is as specified in the subscriber’s content filtering policy.
|
The redirect server may prompt the subscriber to send additional security credentials in order to access the requested content.
|
l
|
terminate-flow: The system gracefully terminates the TCP connection between the subscriber and server, and sends a TCP FIN to the subscriber and a TCP RST to the server.
|
|
l
|
www-reply-code-and-terminate-flow: The system terminates the flow with a specified reply code to the subscriber’s mobile. The reply code is as specified in the subscriber’s content filtering policy.
|
Handling for concatenated and pipelined responses is the same as in Static Content Filtering. The action taken is based on the highest priority category among the pipelined responses.
Content Filtering EDRs are the same as for static rating. However if static rating fails and the request goes for dynamic rating, then Content Filtering EDRs will be generated only after dynamic rating has been completed and not when static rating failed.
If the SRDB task is timed out or some other failure happens, the action configured for failure is taken.
|
16
|
AAA server stops the accounting for the MS for content filtering session and sends Accounting-Stop-Response to the system.
|
How URL Blacklisting and Category-based Content Filtering Work Concurrently
Both URL Blacklisting and Category-based Content Filtering can be concurrently enabled in a system. The following describes how URL blacklisting and content filtering are performed on HTTP/WAP traffic when concurrently enabled on a system:
|
1
|
If both URL Blacklisting and Category-based Content Filtering are enabled, first URL blacklist matching is performed, and then, if required, content filtering is performed.
|
When an HTTP/WAP request comes for ACS processing, a check is made to see if the URL Blacklisting feature is enabled. If enabled, the URL is extracted from the incoming request and is matched with the local Blacklist database.
|
l
|
If a match is found for the URL in the Blacklist database, the packets are subjected to the blacklisting action configured in the rulebase—Discard, Redirect, or Terminate flow. In case of multiple HTTP requests in the same TCP packet, if any of the URLs is blacklisted, then action is taken on the packet.
|
|
l
|
|
l
|
If Category-based Static Content Filtering is enabled, static rating is performed and action taken as configured for the category returned in the subscriber’s content filtering policy.
|
|
l
|
If Category-based Static-and-Dynamic Content Filtering is enabled, first static rating is performed, if the category returned is DYNAM or UNKNOW, dynamic rating is performed, and action taken as configured for the category returned in the subscriber’s content filtering policy.
|
|
2
|
If URL Blacklisting is enabled and Category-based Content Filtering is disabled, and a match is not found for the URL in the Blacklist database, the request is allowed to pass through, and no Content Filtering EDRs are generated for those flows.
|
ECS supports the streamlined ICAP interface to leverage Deep Packet Inspection to enable external application servers to provide their services without performing DPI, and without being inserted in the data flow. For example, with an external Active Content Filtering (ACF) platform.
A high-level view of the streamlined ICAP interface support for external ACF is shown in the following figure.
The system with ECS is configured to support DPI and the system uses this capability for content charging as well.
If a subscriber initiates a WAP (WAP1.x or WAP2.0) or Web session, the subsequent GET/POST request is detected by the DPI function. The URL of the GET/POST request is extracted and passed, along with subscriber identification information and the subscriber request, in an ICAP message to the application server.
In the case of Category-based Content Filtering solution, the application server checks the URL on the basis of its category and other classifications like type, access level and content category and decides if the request should be authorized, blocked, or redirected by answering to the GET/POST with:
|
l
|
A 302 Redirect message in case of redirection. This redirect message includes the URL to which the subscriber should be redirected.
|
Depending on the response received, the system with ECS will either pass the request unmodified, or discard the message, and respond to the subscriber with the appropriate redirection or block message.
Content Charging is performed by the ACS only after the request has been controlled by the application server. This guarantees the appropriate interworking between the external application and content-based billing. In particular, this guarantees that charging will be applied to the appropriate request in case of redirection, and that potential charging based redirections (i.e. Advice of Charge, Top Up page, etc.) will not interfere with the decisions taken by the application server.
|
l
|
Determining the appropriate action (permit, deny, redirect) to take for this type of content based on subscriber profile.
|
|
l
|
For information on configuring the ICAP interface support for external ACF servers, refer to the ICAP Interface Support chapter of the System Enhanced Feature Configuration Guide.
To store generated xDR files, on the ST40 chassis, the system allocates 512 MB of memory on the PSC RAM, and on the ST16 chassis, 256 MB on the PAC RAM. The generated xDRs are stored in CSV format in the /records directory on the PAC/PSC RAM. These generated xDRs can be used for billing as well as for generation of reports to analyze network usage and subscriber trends.
As this temporary storage space (size configurable) reaches its limit, the system deletes older xDRs to make room for new xDRs. Setting gzip file compression extends the storage capacity by approximately 10:1.
Because of the volatile nature of the memory, xDRs can be lost due to overwriting, deletion, or unforeseen events such as power or network failure or unplanned chassis switchover. To avoid loosing charging and network analysis information, configure the CDR subsystem in conjunction with the External Storage System (ESS) to offload the xDRs for storage and analysis.
For more information on the ESS, refer to the ESS Installation and Administration Guide.
This section identifies the minimum system requirements for components of the URL Blacklisting / Category-based Content Filtering solutions.
IMPORTANT: The hardware required for the these components may vary, depending on the number of clients that require access, components managed, and other variables like EDR generation rate or CDR storage and processing requirements.For information on system requirements for the WEM, refer to the WEM Installation and Administration Guide.
IMPORTANT: If you plan to install software and maintain the servers and applications remotely, it is recommended that you use an X-Windows client.Apart from other software requirements for CF-CCI application installation, Java Development Kit 5 is required on the server side.
|
l
|
Workstation supporting Solaris/Sun, Linux, UNIX, Microsoft Windows XP, Windows 2000, or Windows NT operating system
|
|
l
|
Microsoft Internet Explorer version 5.0 (or higher), Netscape Navigator version 4.72 (or higher), or other Internet browser
|
|
l
|
Workstation supporting Solaris/Sun, Linux, UNIX, Microsoft Windows XP, Windows 2000, or Windows NT operating system
|
The ST-series Multimedia Core Platform requires the following additional hardware and memory to handle the Content Rating Master Databases; for example, for Category-based Content Filtering OPTCMDB. The memory required may vary with the size of rating databases used for content rating service.
